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The  following  sections  describe  the  principal  results  of  projects  supported  by  this  contract. 


1  Memory  and  incidental  learning  for  visual  frozen  noise  sequences 

Published  in  Vision  Research  2013,  in  press 

Contributors:  lason  M.  Gold,  Avigael  Aizenman,  Stephanie  Bond  &  Robert  Sekuler 


1 . 1  Project  Summary 

Five  experiments  explored  short-term  memory  and  incidental  learning  for  random  visual  spatio- 
temporal  sequences.  In  each  experiment,  human  observers  saw  samples  of  8  Hz  temporally- 
modulated  ID  or  2D  contrast  noise  sequences  whose  members  were  either  uncorrelated  across 
an  entire  one-second  long  stimulus  sequence,  or  comprised  two  sequences  of  “frozen  noise” 
that  repeated  identically  between  a  stimulus'  first  and  second  500  ms  halves  (Repeated  noise). 
Figure  1  shows  the  three  types  of  stimuli  in  schematic  fashion.  Presented  with  randomly  inter¬ 
mixed  stimuli  of  both  types,  observers  judged  whether  each  sequence  repeated  or  not.  Addi¬ 
tionally,  a  particular  exemplar  of  Repeated  Noise  (a  frozen  or  Fixed  Repeated  noise)  was  inter¬ 
spersed  multiple  times  within  a  block  of  trials. 

As  previously  shown  with  auditory  frozen  noise  stimuli  (Agus  et  al.}  2010)  recognition  per¬ 
formance  {d')  increased  with  successive  presentations  of  a  Fixed  Repeated  stimulus,  and  ex¬ 
ceeded  performance  with  regular  Repeated  Noise.  However,  unlike  the  case  with  auditory 
stimuli,  learning  of  random  visual  stimuli  was  slow  and  gradual,  rather  than  fast  and  abrupt. 
Reverse  correlation  analysis  revealed  that  contrasts  occupying  particular  temporal  positions 
within  a  sequence  had  disproportionately  heavy  weight  in  observers'  judgments.  An  analysis 
of  performance  changes  with  repeated,  occasional  presentation  of  the  same  stimulus  exemplar 
demonstrated  learning,  despite  the  fact  that  such  learning  was  utterly  incidental  to  the  sub¬ 
jects'  actual  task.  A  subsequent  experiment  suggested  that  this  result  arose  from  observers' 
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Figure  1:  Schematic  of  stimulus  sequences  used  in  Project  1.  RefRN  =  Fixed  Repeated  noise 
sequence;  RN  =  Repeated  Noise  sequence. 

uncertainty  about  the  temporal  mid-point  of  the  noise  sequences.  Additionally,  discrimination 
performance  fell  dramatically  when  a  sequence  of  contrast  values  was  repeated,  but  in  reverse 
(“mirror  image”)  order.  This  very  poor  performance  with  temporal  mirror  images  differs  strik¬ 
ingly  from  the  exquisite  sensitivity  that  vision  shows  for  spatial  mirror  images. 

2  Multisensory  integration  in  short-term  memory 

Not  yet  published.  Manuscript  is  in  draft  form. 

Contributors:  Avigael  Aizenman,  Jason  M.  Gold  &  Robert  Sekuler 


2. 1  Project  Summary 

This  effort  extends  the  approach  developed  in  Project  1.  Goals  of  this  Project  were  to  (i)  replicate 
Project  l’s  basic  findings  on  visual  memory,  (ii)  assay  for  analogous  effects  in  the  auditory  do¬ 
main,  (iii)  examine  effects  of  presenting  concurrent  correlated  visual  and  auditory  sequences, 
and  (iv)  examine  the  impact  of  music  training’s  on  subjects’  ability  to  process  rapidly-presented 
stimulus  sequences.  It  is  known  that  music  training  enhances  multiple  aspects  of  hearing,  in¬ 
cluding  ability  to  track  the  temporal  profile  of  complex  sounds,  such  as  speech.  To  determine 
whether  such  training  also  affected  visual  processing  and  responses  to  correlated  audio  and  vi¬ 
sual  signals,  we  assessed  short-term  memory  for  rapidly  presented  sequences  of  quasi-random 
auditory,  visual  or  combined  auditory- visual. 

A  monotonic  correlation  between  luminance  and  pitch  elements  comprising  a  concurrent 
auditory- visual  sequence  not  only  sharpened  perception,  but  produced  an  substantial  improve¬ 
ment  in  short-term  memory  for  the  correlated  inputs.  Significantly,  as  Figure  2  shows,  modest 
amounts  of  training  to  play  an  instrument  enhances  short-term  memory  not  only  for  auditory 
sequences,  but  for  visual  and  audio-visual  sequences  as  well.  This  raises  the  intriguing  ques¬ 
tion  of  whether  music  training  might  have  a  broad  beneficial  impact  on  ability  to  process  and 
recognize  sequences  more  generally. 
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Figure  2:  Principal  results  of  Project  2.  Performance,  expressed  in  the  metric  of  d! ,  is  shown 
separately  for  subjects  who  had  musical  training  and  subjects  who  did  not.  A  =  sequences 
comprising  items  of  varying  pitch;  V  =  sequences  comprising  items  of  varying  luminance; 
AVcon  =  sequences  in  which  concurrent  items  were  monotonically  related  in  luminance  and 
pitch;  AVincon  =  sequences  in  which  concurrent  auditory  and  visual  items  were  uncorrelated 
in  luminance  and  pitch. 

3  Paying  attention  to  attention  in  recognition  memory:  Insights  from 
models  and  electrophysiology 

Published  in  Psychological  Science}  October  25,  2013 

Contributors:  Chad  Dube,  Lisa  Payne,  Robert  Sekuler  &  Caren  M.  Rotello 


3.1  Project  Summary 

Reliance  on  remembered  facts  or  events  requires  memory  for  their  sources,  that  is,  the  con¬ 
texts  in  which  those  facts  or  events  were  embedded.  Understanding  of  source  retrieval  has  been 
stymied  by  the  fact  that  uncontrolled  fluctuations  of  attention  during  encoding  can  cloud  im¬ 
portant  distinctions  between  competing  theoretical  accounts.  To  clarify  the  issue,  we  combined 
electrophysiology  (high-density  EEG  recordings)  with  computational  modeling  of  behavioral 
results.  We  manipulated  subjects’  attention  to  an  auditory  attribute,  whether  the  source  of  indi¬ 
vidual  study  words  was  a  male  or  female  speaker.  Posterior  alpha  band  (8-14  Hz)  power  in  sub¬ 
jects’  EEG  increased  after  a  cue  to  ignore  the  gender  of  the  person  who  was  about  to  speak.  With 
control  of  subjects’  attention  validated  by  the  pattern  of  EEG  signals,  computational  modeling 
showed  unequivocally  that  memory  for  source  (male  or  female  speaker)  reflects  a  continuous, 
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signal  detection  process  rather  than  a  threshold  recollection  process.  Unfortunately,  we  were 
not  able  to  test  enough  subjects  to  support  an  examination  of  individual  differences  in  ability 
to  filter  out  extraneous  sources  of  information.  Clearly,  that  would  be  a  useful  follow  up  to  this 
project. 

4  The  importance  of  ignoring:  Alpha  oscillations  protect  selective  pro  - 
cessing 

Published  in  Current  Directions  in  Psychological  Science ,  in  press. 

Contributors:  Lisa  Payne  &  Robert  Sekuler 

4. 1  Project  Summary 

Selective  attention  is  often  thought  to  entail  an  enhancement  of  some  task-relevant  stimulus 
or  attribute.  We  discuss  a  somewhat  different  perspective,  namely  that  ignoring  irrelevant,  dis¬ 
tracting  information  plays  a  complementary  role  in  information  processing.  Cortical  oscilla¬ 
tions  within  the  alpha  (8-14  Flz)  frequency  band  can  serve  as  a  marker  of  sensory  suppression. 
This  suppression  is  linked  to  selective  attention  for  visual,  auditory,  somatic,  and  verbal  stimuli. 
Inhibiting  irrelevant  processing  allows  us  to  make  more  accurate  and  timely  responses.  It  also 
helps  protect  material  that  we  hold  in  short-term  memory  from  being  disrupted.  Furthermore, 
this  selective  process  keeps  irrelevant  information  from  distorting  the  precision  of  memories. 
At  best,  human  memory  is  only  as  good  as  the  perceptual  representations  on  which  it  depends, 
and  on  the  ability  to  maintain  those  representations  in  useful  form.  Modulation  of  cortical  alpha 
oscillations  can  be  exploited  as  an  active,  purposeful  mechanism  to  help  people  pay  attention 
and  remember  the  things  that  matter. 

5  Similarity-based  distortion  of  short-term  visual  memory  is  percep¬ 
tual 

Published  in  Vision  Research}  2013,  in  press 

Contributors:  Chad  Dube,  Feng  Zhou,  Michael  J.  Kahana  &  Robert  Sekuler 


5.1  Project  Summary 

A  task- irrelevant  stimulus  can  distort  recall  from  visual  short-term  memory.  Specifically,  the 
recalled  stimulus  can  take  on  some  of  the  appearance  of  an  accompanying  task- irrelevant  stim¬ 
ulus  (Huang  &  Sekuler,  2010).  However,  the  locus  of  this  robust  effect  is  unknown  as  it  is  unclear 
whether  distortion  is  generated  early,  during  perceptual  encoding  of  target  stimuli,  or  later,  dur¬ 
ing  some  post-perceptual  representation  in  memory.  To  select  between  these  two  accounts,  we 
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manipulated  subjects’  attention  to  target  and  non-target  stimuli  whose  similarity  relationships 
were  varied  parametrically.  On  each  trial,  subjects  were  shown  one  or  two  Gabor  patches,  fol¬ 
lowed  by  a  probe;  their  task  was  to  indicate  whether  the  probe  matched  one  of  the  study  items. 
A  brief  cue  told  subjects  which  Gabor,  first  or  second,  would  serve  as  that  trial’s  target  item. 
Critically,  this  cue  appeared  either  before,  between,  or  after  the  study  items. 

False  alarm  rates  varied  substantially  with  the  probe’s  similarity  to  a  trial’s  task- irrelevant 
study  item,  confirming  that  the  task-irrelevant  item  impacted  visual  encoding  of  the  target.  A 
drift-diffusion  model  (Ratcliff,  1978)  was  applied  simultaneously  to  subjects’  choice  probabili¬ 
ties  and  reaction  times.  Model  fits  revealed  that  both  lure  similarity  and  cue  timing  affect  the 
model’s  mean  drift  rate  parameter,  which  represents  the  accumulation  of  sensory  information. 
This  suggests  that  selective  attention  in  the  pre-cue  condition  operated  on  the  encoding  of  tar¬ 
get  stimuli,  consistent  with  a  perceptual  account  of  the  distortion’s  origin.  The  publication  dis¬ 
cusses  the  implications  of  these  results  for  theories  of  visual  short-term  memory. 

6  Aging  and  audio-visual  and  multi-cue  integration  in  motion 

Published  in  Frontiers  in  Psychology ,  May  23,  2013 

Contributors:  Eugenie  Roudaia,  Allison  B.  Sekuler,  Patrick  1.  Bennett  &  Robert  Sekuler 


6. 1  Project  Summary 

The  perception  of  naturalistic  events  relies  on  the  ability  to  integrate  information  from  multi¬ 
ple  sensory  systems,  an  ability  that  may  change  with  healthy  aging.  When  two  objects  move 
toward  and  then  past  one  another,  their  trajectories  are  perceptually  ambiguous:  the  objects 
may  seem  to  stream  past  one  another  or  bounce  off  one  another.  Previous  research  showed  that 
auditory  or  visual  events  that  occur  at  the  time  of  disks’  coincidence  could  bias  the  percept  to¬ 
ward  bouncing  or  streaming.  We  exploited  this  malleable  percept  to  assay  age-related  changes 
in  the  integration  of  multiple  inter-  and  intra-modal  cues.  The  disks’  relative  luminances  were 
manipulated  to  produce  stimuli  strongly  favoring  either  bouncing  or  streaming,  or  to  produce 
ambiguous  motion  (equal  luminances).  A  sharp  sound  coincident  with  the  disks’  overlap  in¬ 
creased  both  groups’  perception  of  bouncing,  but  did  so  significantly  less  for  older  subjects. 
An  occluder’s  impact  on  motion  perception  varied  with  its  duration:  a  long  duration  occluder 
promoted  streaming  in  both  groups;  a  brief  occluder  promoted  bouncing  in  younger  subjects, 
but  not  older  ones.  Control  experiments  demonstrated  that  the  observed  differences  between 
younger  and  older  subjects  resulted  from  neither  age-related  changes  in  retinal  illuminance  nor 
age-related  changes  in  hearing,  pointing  to  weakened  inter-  and  intra-modal  integration  with 
aging.  These  changes  could  contribute  to  previously  demonstrated  age-related  perceptual  and 
memory  deficits. 
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Figure  3:  Schematic  of  conditions  used  in  Project  8.  Stimuli  were  Gabors  whose  spatial  fre¬ 
quency  was  varied;  two  Gabor  study  items  were  briefly  presented  on  each  trial.  These  were 
followed  by  a  test  Gabor  whose  spatial  frequency,  on  half  the  trials,  matched  that  of  one  of 
the  study  items.  Subjects  were  instructed  to  ignore  location  and  judge  whether  there  was  a 
match  solely  on  spatial  frequency. 

7  Age-Related  Changes  in  Expectation-Based  Modulation  of  Motion 
Detectability 

Published  in  PLoS  One ,  August  9,  2013 

Contributors:  Theodore  P.  Zanto,  Robert  Sekuler,  Chad  Dube  &  Adam  Gazzaley 


7. 1  Project  Summary 

Expecting  motion  in  some  particular  direction  biases  sensitivity  to  that  direction,  which  speeds 
detection  of  motion.  However,  the  neural  processes  underlying  this  effect  remain  underex¬ 
plored,  especially  in  the  context  of  normal  aging.  To  address  this,  we  examined  younger  and 
older  adults'  performance  in  a  motion  detection  task.  In  separate  conditions,  the  probability 
was  either  50%  or  100%  that  a  field  of  dots  would  move  coherently  in  the  direction  a  participant 
expected  (either  vertically  or  horizontally).  Expectation  and  aging  effects  were  assessed  via  re¬ 
sponse  times  (RT)  to  detect  motion  and  electroencephalography  (EEG).  In  both  age  groups,  RTs 
were  fastest  when  motion  was  similar  to  the  expected  direction  of  motion.  RT  tuning  curves 
exhibited  a  characteristic  [/-shape  such  that  detection  time  increased  with  an  increasing  devi¬ 
ation  from  the  participant's  expected  direction.  Strikingly,  EEG  results  showed  an  analogous, 
hyperbolic  curve  for  Nl  amplitude,  reflecting  neural  biasing.  Though  the  form  of  behavioral 
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and  EEG  curves  did  not  vary  with  age,  older  adults  displayed  a  clear  decline  in  the  speed  of  de¬ 
tection  and  a  corresponding  reduction  in  EEG  Nl  amplitude  when  horizontal  (but  not  vertical) 
motion  was  expected.  Our  results  suggest  that  expectation-based  detection  ability  varies  with 
age  and,  for  older  adults,  also  with  axis  of  motion. 

8  What  is  where’s  influence  on  short-term  visual  recognition? 

Not  yet  published.  Manuscript  is  in  draft  form. 

Contributors:  lie  Huang,  Chad  Dube  &  Robert  Sekuler 


8. 1  Project  Summary 

Recognition  of  an  object  is  undermined  if  that  object  appears  in  an  unusual  spatial  context. 
Thus,  an  object’s  “where”  affects  recognition  of  its  “what.”  This  effect  can  persist  even  when 
location  should  be  irrelevant.  To  examine  the  linkage  between  memory  for  “what”  and  “where,” 
we  isolated  each  dimension’s  influence  on  recognition,  using  Gabor  stimuli  as  study  items  and 
probes.  We  varied  (i)  a  probe’s  featural  similarity  to  the  study  items,  and,  at  the  same  time  (ii) 
the  similarity  of  the  probe’s  spatial  location  to  the  spatial  locations  of  the  study  items.  These 
manipulations  are  illustrated  in  Figure  3. 
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Items  in  Memory 


Test  Trial  n 
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Figure  4:  Diagram  illustrating  possible  models  by  which  the  nominally  task-irrelevant  stimulus 
attribute  (location)  might  influence  subjects'  performance  in  Project  8.  P(T|Location)  repre¬ 
sents  the  probability  that  the  target  (test  Gabor  whose  spatial  frequency  matches  that  of  one 
of  the  study  Gabors)  appears  in  a  given  location. 
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A  mismatch  between  the  location  of  the  probe  and  its  most  similar  study  item  exerted  a 
graded  influence  that  spread  over  considerable  distance.  In  turn,  this  graded  influence  strongly 
modulated  the  impact  of  the  featural  similarity  between  probe  and  study  items.  Application 
of  the  Drift  Diffusion  Model  (DDM)  to  reaction  times  and  recognition  responses  supported  this 
interpretation.  The  DDM  analysis  revealed  multiplicative  effects  of  context  and  similarity  on  the 
rate  of  evidence  accumulation:  accumulation  rates  varied  with  featural  similarity,  an  effect  that 
was  strongest  for  probes  tested  at  locations  farther  from  the  best-matching  study  items.  Finally, 
our  results  demonstrate  that  short-term  visual  recognition  entails  a  global  matching  process  of 
the  sort  proposed  in  Sekuler  &  Kahana  (2007). 
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